Papers with text-to-image synthesis
BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Recent text-to-image models require multiple passes of prompt engineering by humans to produce satisfactory results for real-world applications. |
| Approach: | They propose a deep generative model to generate high-quality prompts from raw descriptions using visual feedback. |
| Outcome: | The proposed model produces high-quality prompts from simple raw descriptions . it can be integrated to a cloud-native AI platform to provide better image generation service in the cloud. |
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing tools for text-to-image synthesis can visualize machine imaginations for a given context. |
| Approach: | They propose a framework that uses machine-generated images to guide language models in open-ended text generation. |
| Outcome: | The proposed framework is effective on open-ended text generation tasks while showing minor degeneration. |
Enhancing Multimodal Named Entity Recognition through Adaptive Mixup Image Augmentation (2025.coling-main)
Copied to clipboard
| Challenge: | Current named entity recognition methods struggle with text-image mismatch problem due to a lack of visual context. |
| Approach: | They propose an adaptive mixup image augmentation method that generates augmented images based on matching score between text and image . |
| Outcome: | The proposed method can be integrated into existing models and demonstrate consistent performance improvements. |
Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance (2025.emnlp-main)
Copied to clipboard
Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Justin Li, Jian Gao, Zhang Li, Hao Yang, Boxing Chen, Osamu Yoshie
| Challenge: | Existing solutions for text-to-image synthesis are sensitive on textual prompts, posing a challenge for novice users. |
| Approach: | They propose a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. |
| Outcome: | The proposed model emphasizes user experience for novice users . it improves user-centricity score while maintaining a competitive quality of synthesized images. |
Integrating Visuospatial, Linguistic, and Commonsense Structure into Story Visualization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on text-to-image synthesis does not explore the use of linguistic structure of the input text. |
| Approach: | They propose to use constituency parse trees to encode structured input and a Transformer-based recurrent architecture to augment commonsense information to generate visual story. |
| Outcome: | The proposed model improves visual quality and consistency of images from a target domain without fine-tuning. |